Summary and Contributions: Neural network based compressors usually apply additive uniform noise during training as a proxy for the quantization that is performed during test-time. This creates a mismatch between the training and testing phases. This work proposes to instead apply universal quantization at test time thus eliminating the mismatch between training and test phases while maintaining a differentiable loss function. It is based on the fact that adding uniform noise to an input x is equivalent to subtracting a uniform random variable from x, rounding the result and then adding the same uniform random variable back. As a result, by sharing a random seed across the encoder and decoder we can easily implement universal quantization for neural network based compressors.